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MICRO-CONTROLLABLE,  MULTI-FUNCTIONAL  INTERFACE  MODULE 
FOR  DIGITAL  MP:  A  WEARABLE  COMPUTER  SECURITY  APPLICATION 


SUMMARY 


The  objective  of  this  work  was  the  creation  of  a  wearable  human-computer  interface 
system.  This  system  is  intended  to  provide  the  dismounted  soldier  with  a  hands-free, 
light  weight,  display,  camera,  and  audio  interface  to  a  wearable  computer.  The  display 
consists  of  a  miniature  flat  panel  LCD  and  optical  system  which  magnifies  it  to  create  a 
16-20°  wide  image  at  a  comfortable  viewing  distance.  The  display  is  mounted  on  or  in 
eyeglasses  for  hands-free  operation,  and  provides  clear  lines  of  sight  around  the  display 
to  allow  the  user  to  perform  his  duties.  The  headset  also  incorporates  a  hi^  resolution 
camera  for  reconnaissance  and  capture  of  images  for  face  recognition.  The  user  can 
communicate  with  the  computer  using  either  audio,  including  voice  command  and 
control,  or  with  a  pointing  device  for  more  discrete  operation.  An  industrial  design  effort 
was  undertaken  to  ensure  the  ergonomics  of  the  final  headset  as  well  as  aesthetic 
acceptability.  Finally,  the  user  interface  software  running  on  the  wearable  computer  was 
optimized  for  operation  in  a  wearable  application. 
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1  INTRODUCTION 


The  objective  of  this  woric  was  the  creation  of  a  wearable  human-computa*  interface 
system.  This  system  is  intended  to  provide  the  dismounted  soldier  with  a  hands-free, 
light  weight,  display,  camera,  and  audio  interface  to  a  wearable  computer.  In  this  section 
we  review  the  contract  objectives  and  outcome.  Details  are  provided  in  the  sections  to 
follow. 

This  work  is  part  of  a  joint  effort  between  The  MicroOptical  Corporation,  ViA,  Inc.,  the 
U.S.  Army  Soldier  Systems  Center,  and  the  Military  Police  School  at  Ft.  Polk,  LA 
MicroOptical’s  tasks  involved  the  development  of  the  head-mounted  interface  system, 
ViA  was  responsible  for  the  wearable  computer.  The  Soldier  Systems  Cento  provided 
human  factors  analysis  and  ovemll  project  integration.  The  Military  Police  School 
provided  us^  anal^is  and  field  trials.  This  report  covers  only  the  development  of  the 
eyeglass  interface  system,  which  is  termed  the  micro-controllable  multi-fimctional 
interface  module  (MMIM).  The  ^tire  system  including  computer,  peripherals,  and 
MMIM  has  been  termed  the  “Digital  MP.” 


A  large  number  of  subcontractors  participated  in  the  creation  of  the  MMIM  system.  The 
subcontractors  included:  Optikos  and  MSOD  for  optical  design;  Stratos  Product 
Development  Group  for  industrial  design;  Aox  Inc.  for  ffiEE  1394  circuit  development, 
MicroDisplay  Corporation  for  the  liquid  crystal  display;  Tris^  Systems,  Inc.  for 
videometric  head  tracking;  and  Randolph  Engineering  for  eyeglass  flame  development. 
In  the  final  s^tem  revision,  Visionics  (now  Identix,  Inc.)  was  subcontracted  to  revise  the 
face  recognition  portion  of  the  software  to  include  automatic  image  capture  and  face 
recognition  capability.  MicioOptical’s  effort  was  managed  by  Mark  Spitzer  and  Paul 
Zavracky.  Contributors  at  MicroOptical  included  Rob^  McClelland,  Paul  Aquilino, 
Noa  Reusing,  Mark  Olson,  and  John  Crawford. 


1.1  PROJECT  OBJECTIVE 

The  purpose  of  the  MicroOptical  work  is  development  and  demonstration  of  an  eyeglass- 
based  MMIM  interface  system  which  would  provide  an  interface  for  all  of  the  functions 
of  the  Digital  MP  system.  These  functions  include: 

1.  Face  recognition  -  in  which  die  head-mount^  camera  is  aimed  at  an  unknown 
person,  the  person’s  face  is  photographed  and  compared  to  a  database,  and 
information  returned  fiom  the  database  is  shown  on  the  head-mounted  display. 

2.  Route  Recon  -  in  which  the  head-mounted  camera  is  used  to  photograph  and  store  the 
images  of  objects  during  route  reconnaissance. 

3.  Peer-to-pe^  conferencing  -  in  which  members  of  a  team  share  audio  and  video. 


2 


4.  Voice  command  -  in  which  spoken  commands  are  used  to  control  the  computer 
operation. 

5 .  Head-tracking  -  for  future  applications  involving  augmented  reality. 

6.  Legacy  compatibility  -  in  which  the  system  shows  images  fiom  FBCB2  sources. 

These  functional  requirements  placed  hardware  requirements  on  all  of  the  subsystems. 
Specifications  for  each  subsystem  are  discussed  in  Section  1.2. 

The  first  objective  of  the  program  was  to  develop  the  technology  that  would  make  such  a 
system  possible.  The  second  project  objective  was  to  integrate  the  technologies  and 
demonstrate  a  working  system.  The  final  objective  was  to  deliver  working  systems  to  the 
Soldier  Center  for  testing  at  Ft,  Polk.  All  objectives  were  obtained. 

1.2  MMIM  SYSTEM  REQUIREMENTS 

The  above  system  functions  required  the  development  of  specifications  for  the 
subsystems  of  the  MMIM  system.  Table  1  summarizes  the  initial  specifications  for  the 
display  system.  The  camera  system  is  specified  in  Table  2.  Audio  is  specified  in  Table  3. 


Table  1.  Display  Specifications 


Resolution 

640x480  pixels  or  greater,  60  Hz  refiresh 

Color  depth 

18  bit  color 

Cable  length 

4  ft  firom  viewer  to  belt  worn  controls 

Field  of  view 

20°  Horizontal,  15°  Diagonal 

Focus  range 

2m 

Table  2.  Camera  Specifications 


Sensor 

640x480  pixels,  1/3”  active  area  color  CMOS  sensor 

15  fps  @  640x480,  30  fps  @  320x240 

Cable  length 

4  ft  from  eyeglasses  to  control  box 

Field  of  view 

Approximately  20°  Horizontal 

Distortion 

<2% 

Focus 

Adjustable,  2ft  to  infinity 

Table  3.  Audio 


Microphone 

Noise  canceling,  flat  frequency  response  to  5  KHz 

Digitization 

llKHz 

Speaker 

Monophonic 
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1.3  RESULTS 


Figure  1  shows  a  photograph  of  the  completed  MMIM  system.  Systems  meeting  the 
above  specifications  were  delivered  for  testing  at  Ft.  Polk,  and  the  result  is  summarized  in 
Section  7  of  this  report.  Based  on  the  responses  to  the  field  testing,  the  hardware  and 
software  were  revised  as  described  in  Section  8. 


Figure  1,  Completed  MMIM  System  with  SVGA  display 


2  SVGA  Display  Subsystem 


The  display  subsystem  comprises  two  main  parts.  The  first  is  the  active  matrix  liquid 
crystal  display  (AMLCD),  and  the  second  is  the  optical  system  for  illuminating  and 
viewing  the  display. 

2.1  AMLCD  Development 

The  MMIM  system  requires  a  small  liquid  crystal  display  capable  of  integration  in 
eyewear.  To  this  end,  Microdisplay  Corporation  began  the  layout  of  a  new  reflective 
AMLCD.  Although  the  original  system  specification  called  for  VGA  (640x480) 
resolution.  Microdisplay  Corporation  offered  to  build  an  SVGA  (800x600)  display  for  the 
project.  The  pixel  pitch  was  approximately  10  ^m,  and  thus  the  viewable  area  of  the 
display  was  8  mm  by  6  mm. 

Color  was  obtained  by  using  the  time  sequential  color  firame  method.  Thus,  to  obtain  a 
fi-ame  rate  of  60  Hz,  the  display  was  run  at  a  fi*ame  rate  of  1 80  Hz  which  was  sub-divided 
into  red,  green  and  blue  frames.  The  use  of  this  technique  required  a  frame  buffer. 
Integration  of  a  frame  buffer  within  the  display  was  not  feasible;  therefore,  complete 
integration  using  a  red/green/blue  (RGB)  signal  as  input  was  not  possible.  However, 
cable  was  developed  that  enabled  the  display  to  be  connected  conveniently  to  drive 
circuitry  mounted  off  the  head. 

Fig.  2  illustrates  the  front  and  back  view  of  the  display.  The  AMLCD  is  mounted  to  a 
flex-PC  board  which  terminates  in  a  connector  matching  the  cable. 


Figure  2.  Photograph  of  the  Front  and  Back  of  the  AMLCD 
Developed  in  this  Work. 


Experimental  drive  electronics  for  the  display  were  provided  by  Microdisplay.  Fig.  2 
illustrates  one  of  the  sets  of  electronics.  The  input  to  die  circuit  was  conventional  analog 
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RGB.  Thus  it  was  necessary  to  develop  additional 
drive  electronics  for  IEEE  1394  input.  Initial  units 
wa*e  delivered  to  the  Army  for  evaluation  using  this 
type  of  electronic  drive  circuitry. 

Figure  3  shows  an  electronics  package  created  by 
MicroOptical  using  the  Microdisplay  drive  board,  and 
a  USB  camera  drive  board.  This  allowed  connection 
to  the  ViA  computer  by  conventional  techniques. 
Later  units  demonstrated  IEEE  1394  drives;  however, 
we  found  that  the  Microsoft  Windows  2000  operating 
system  is  not  fiilly  compatible  with  an  IEEE  1394 
display  drive.  While  a  Firewire  display  is  supported 
by  the  operating  system,  it  cannot  use  it  as  a  primary 
display.  Thus  the  development  of  the  IEEE  1394 
systems  and  the  required  ASIC  was  not  fully 
implemented. 


Figure  3.  Photograph  of  Drive 
Eleetronics  for  Initial  Display  and 
Camera. 


2.2  Optical  System 

The  optical  system  comprises  and  illuminator  and  a  magnifier. 

2.2.1  Illuminator 

The  AMLCD  fabricated  by  Microdi^lay  require  a  fi-ont  illumination  system.  The 
illumination  system  provides  collimated  polarized  li^t  to  the  display,  and  analy^s  the 
reflected  light  in  a  second  polarizer. 

A  number  of  approaches  to  analyzing  the  polarized  light  fix)m  the  reflected  display  are 
possible,  including  the  use  of  film  polarizere  made  by  Polaroid  (now  a  division  of  3M). 
The  optical  design  also  included  high  quality  thin  film  polarizing  beam  splitter  coatings 
designed  specifically  for  our  requirements  and  that  were  deposited  on  glass  prisms.  This 
provided  us  with  the  highest  quality  image;  however,  the  wei^t  of  the  system  was 
inerted  by  the  use  of  solid  glass  cubes. 
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2.2.2  Integrated  Magnifier 


Two  types  of  magnifying  optical  systems  were  constructed  in  this  program.  The  first  was 
an  integrated  eyeglass  lens  which  used  a  polarizing  cube  illuminator  as  described  above 
and  a  polarizing  eyepiece  to  allow  the  user  to  see  through  the  eyepiece  when  the  display 
is  not  in  use.  This  eyepiece  is  shown  in  Fig.  4.  Note  that  the  illuminator  is  external  to 
the  eyeglass  lens.  This  allows  the  introduction  of  a  field  lens  to  improve  the  image. 


Figure  4.  Integrated  Magnifying  ‘^See-through”  Eyepiece. 

Eyewear  Display  were  formed  using  the  lens  shown  in  Fig.  4.  Note  that  this  is  a  see- 
through  eyepiece  design.  The  reason  see-through  is  preferred  in  an  integrated  optical 
approach  is  that  the  user  cannot  move  the  optics  out  of  the  way  when  using  the  eyewear 
with  the  display  turned  off.  If  this  were  not  a  see-through  system,  the  user  must  remove 
the  eyewear  to  remove  the  dark  spot  in  his  vision.  Thus  see-through  optics  allow 
multiple  uses  of  the  eyewear,  including  augmented  reality  systems  in  which  information 
is  overlayed  through  the  see-through  eyepiece.  Fig.  5  shows  a  photograph  of  the 
eyewear.  The  industrial  design  that  went  into  the  creation  of  the  eyewear  in  Fig.  5  will  be 
discussed  in  Section  5. 
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Figure  5.  Photograph  of  Integrated  Eyewear  With 
Reflective  SVGA  Display,  The  Horizontal  FOV  is  16®, 


2.2.3  Clip-on  Magnifier 

The  clip-on  version  of  this  system  is  illustrated  in  Fig.  6.  These  optics  are  not  see 
through,  owing  to  the  location  of  the  mirror.  This  design  was  chosen  because  the  user 
can  move  the  clip-on  out  of  his  field  of  view  when  it  is  not  in  use*.  The  use  of  occluding 
optics  of  this  type  provides  higher  contrast  in  high  brightness  ambient  light  environment. 
Fig.  7  shows  a  photograph  of  the  final  display  system,  built  in  a  clip-on  form 


BEAM  SPLITTER 


Figure  6.  Clip-on  Magnifier  and  Dluminator 
System. 


Figure  7.  Photograph  of  the  SVGA  Clip-on 
Display. 


3  Camera  Subsystem 


The  camera  subsystem  comprises  three  main  parts:  (1)  the  lens,  (2)  the  CMOS  imager 
integrated  circuit,  and  (3)  the  interface  electronics.  The  design  of  the  lens  is  related 
highly  to  the  specifications  of  the  CMOS  chip,  owing  to  exposure  requirements,  head 
motion,  and  other  factors. 


3.1  Lens  Design 

The  lens  design  was  carried  out  by  Optikos.  The  initial  assumptions  were  the  following: 

(1)  VGA  (640x480)  Image  Sensor 

(2)  9.0  fim  Square  Pixel 

(3)  7.2  mm  Sensor  Diagonal 

(4)  30  pixels  Between  Eyes  for  Face  Recognition 

The  lens  design  task  comprised  a  determination  of  the  exposure,  ^erture,  resolution  and 
head  motion  requirements  for  the  camera.  The  criteria  imposed  on  the  design  was  to 
provide  the  smallest  possible  system  consistent  with  indoor  li^t  levels,  head  motion 
characteristic  of  a  person  holding  the  head  reasonably  still  (<1  7s  rotation) ,  and  which 
provided  a  640x480  image  with  pixel  blur  of  1  pixel. 

Initial  work  examined  the  depth  of  focus  that  could  be  provided  for  different  ^erture 
stops.  In  each  zone,  once  the  focus  is  set,  the  image  is  in  focus  for  all  positions  within  the 
zone.  An  optical  system  with  a  single  zone  would  not  need  any  focus  adjustment; 
however,  a  camera  with  such  a  hi^  depth  of  focus  requires  an  unacceptably  small 
aperture,  owing  to  die  sensitivity  range  of  current  CMOS  sensors. 

The  initial  lens  design  was  studied  for  two  apertures  yielding  f/3  and  f/l  systems  and  the 
resulting  focus  zones  are  as  follows: 

(1)  f/3:  5.8  fl  to  8.8  fl,  9  fl  to  17.5  ft,  and  17.5  ft  to  infinity 

(2)  f/l:  3.75  ft  to  7.5  ft,  and  7.5  ft  to  infinity 

Thus  with  the  smaller  ^erture,  the  camera  had  two  simple  focus  zones  in  which  VGA 
resolution  was  obtainable  for  the  entire  zone.  Owing  to  the  depth  of  field  and  field  of 
view  (20°  horizontal),  the  lens  design  effort  showed  that  a  person  would  have  to  be  within 
12.9  ft  to  have  30  pixels  between  the  eyes  and  thus  to  be  recognized  by  face  recognition 
software. 

Exposure  calculations  were  carried  out.  For  a  sensor  integration  of  33  ms,  at  f/3  the 
systems  requires  33.4  foot  candles,  and  for  f/7  the  system  requires  18.4  foot  candles. 
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Blur  calculations  wore  carried  out  to  determine  the  effect  of  head  rotation  to  the  left  or 
right.  The  horizontal  field  of  view  of  2(f  translates  to  and  angle  of  0.03  degrees  per 
colunm.  Therefore  there  will  be  pixel  blur  of  one  pixel  if  the  head  moves  at  a  rate  of 
greater  than  1  ®/s. 


3,2  CMOS  Sensor  Selection 

The  selection  of  a  CMOS  sensor  was  based  on  a  requirement  for  VGA  resolution  and 
exposure  times  that  would  permit  a  degree  of  head  motion  in  daylight  situations.  A 
CMOS  sensor  was  selected  over  a  CCD  sensor  owing  to  the  desire  to  reduce  the  power 
consumption  to  the  lowest  levels  possible,  and  to  the  belief  that  eventually  CMOS 
devices  would  outperform  CCD  sensors,  ultimately  at  lower  prices.  We  examined  two 
principal  manufacturers:  Hewlett-Packard  and  Hyun^, 

A  preliminary  data  sheet  for  the  Hyundai  CMOS  imager  was  obtained  and  compared  with 
five  other  similar  devices,  including  HP  CMOS  sensors.  The  Hyundai  imager  which  was 
recommended  by  Aox  was  determined  to  be  the  best  fit  to  our  requirements.  The 
Hyundai  device  has  a  built-in  eight  bit  aimlog  to  digital  converter. 

To  assess  the  performance  of  the  Hyun^  sensor  in  a  face  recognition  application,  we 
have  made  a  calculation  of  the  resolution  for  a  s^tem  with  an  18  degree  field  of  view 
(FOV).  Table  4  shows  the  performance  of  the  640x480  camera  sensor  in  such  a  sj^tem. 
It  can  be  seen  that  at  5  feet  the  target  width  captured  by  the  camera  is  L56  feet.  An  object 
in  this  focal  plane  at  5  feet  is  resolved  by  34  lines  per  inch.  A  key  factor  in  the 
performance  of  a  face  recognition  system  is  the  number  of  lines  between  the  eyes.  If  we 
assume  that  the  typical  inteipupillary  distance  is  65  mm,  we  find  that  at  five  feet,  the 
system  provides  87  lines  which  corre^onds  to  a  measurement  of  IPD  of  better  ttoi  1 
mm.  We  believe  that  even  at  10  ft  the  syst^  should  have  adequate  resolution  for  face 
recognition. 


Table  4.  Hyundai  Device  With  18  D^ree  FOV  Optics 


Target 

Distance 

(feet) 

Target 

Size 

(feet) 

Vertical 
Lines 
Per  Inch 

Vertical 
Lines 
per  65  mm 

3 

0.94 

57 

145 

5 

1.56 

34 

87 

10 

3.13 

17 

44 

15 

4.69 

'  11 

29 

25 

7.82 

7 

17 

Note:  Horizontal  re^lution  is  640  lines. 


The  heart  of  the  camera  syst^  is  a  CMOS  sensor  capable  of  VGA  resolution.  The  sensor 
is  shown  in  its  package  and  mounted  to  a  circuit  board  in  Fig.  3.  Also  shown  in  the 


10 


figure  are  the  optics,  which  have  a  focal  length  of  17  mm  and  fi'3.5.  The  camera  optics 
were  designed  for  face  recognition  by  Optikos,  and  through  considerations  related  to  this 
recognition,  we  determined  that  the  field  of  view  should  be  19®  horizontal  by  14.5° 
vertical.  Distortion  is  less  than  2%.  The  camera  has  a  focus  range  of  2  ft  to  infinity,  and 
is  provisioned  for  servo  focusing.  The  weight  of  the  camera  is  37  grams. 


3.3  Interface  Electronics 


Development  of  interface  electronics  comprised 
two  parts:  (1)  initial  development  of  USB  drive 
electronics,  and  final  development  of  IEEE  1394 
drive  electronics. 

The  USB  drive  electronics  uses  an  integrated 
circuit  developed  by  Aox.  This  IC  was  used  in  a 
board  designed  by  MicroOptical  and  shown  in  Fig. 
8.  This  interface  was  used  in  the  development  and 
testing  of  the  camera. 


3.4  Camera  Subsystem  Initial  Design 


Figure  8.  USB  Interface  Electronics 
Using  an  Aox  Integrated  Circuit 


Fig.  9  shows  a  solid  model  illustration  of  the 
mechanical  design  of  the  camera.  Note  that  a 
prism  is  used  to  turn  the  image  fi’om  the  lens  barrel 
into  the  CMOS  sensor.  A  picture  of  the  packaged 
CMOS  sensor  is  shown  in  Fig.  10.  The  use  of  a 
turning  prism  moxmted  on  top  of  the  CMOS  sensor 
allows  ihQ  camera  to  be  made  more  compact,  and 
also  allows  a  reduction  in  the  back  focal  length  of 
the  camera  lens  system. 


Figure  9.  Solid  Model  of  the  Initial 
Camera  Design. 


The  lens  is  fitted  with  a  removable  aperture  so  that  the  camera  could  be  tested  with  two  f- 
stops.  Consistent  with  the  design  described  above,  the  hi^  number  of  surfaces  in  the 
camera  lens  creates  an  undistorted  image  capable  of  VGA  resolution  and  very  well  suited 
for  face  recognition.  A  photograph  if  the  initial  camera  design  is  shown  in  Fig.  1 1 . 
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Figure  10.  CMOS  Sensor,  Focus 
Servo  and  Servo  Drive  Circuit. 


Figure  11.  Photograph  of  Camera 
Made  in  Accordance  with  the  Initial 
Camera  Desian. 


3,5  Camera  Testing 


The  camera  was  tested  using  the  USB  interface  with  Face-It™  software  to  determine  how 
well  it  performed  in  a  face  recognition  task.  Figure  12  shows  the  results  of  testing  at 
three  distances  from  the  camera.  The  software  is  able  to  recognize  the  subject  at  5  and  10 
feet. 


Five  Feet 


Ten  Feet 


40  FBE 


21PBE 


Score  8.4 


Score  8,3 


Fifteen  Feet 
14PBE 
Not  matched 


Figure  12  .  Outdoor  Testing  of  Camera  Using  Face  Recognition  Software  (in 
shade,  f/7,  using  the  center  320x240  area). 


The  camera  was  mounted  on  a  rotation  platform  to  simulate  head  motion.  Figure  13 
shows  the  results  of  testing  the  camera  in  this  way.  Note  that  for  rotation  rates  above  5 
degrees  per  second,  the  camera  was  not  able  to  capture  a  reliable  image.  This  shows  the 
need  for  good  lighting  in  face  recognition  applications.  Such  lighting  reduces  the 
exposure  time  for  the  camera  sensor  and  reduces  blur. 
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77s 


Still 


57s 


Score  93  Score  8.0  Score  73 

Figure.  13  .  Camera  Testing  on  Rotation  Platform. 


87s 

Score  6.7 


3.6  Camera  Subsystem  Final  Design 

After  gathering  some  experience  in  using  the  camera,  it  was  decided  that  the  performance 
of  the  f/#  7  system  was  preferable  in  most  situations,  due  to  its  greater  depth  of  focus. 
While  the  faster  lens  offers  somewhat  better  low  light  performance,  the  sensitivity  of  the 
sensor  is  such  that  in  most  situations  the  ft#7  performance  was  sufficient.  By  eliminating 
the  ft#  3  option  and  redesigning  for  a  smaller  lens  diameter  we  both  reduced  the  physical 
size  of  the  camera  and  simplified  its  operation.  At  ray  trace  of  the  final  design  is  shown  in 
Fig,  14  .  Note  that  the  use  of  a  solid  prism  allows  a  shortening  of  the  physical  path  length 
as  compared  to  the  focal  length.  The  transfer  function  for  this  optical  system  is  shown  in 
Fig.  15.  Fig.  16  shows  the  measurement  results  of  the  modulation  transfer  function. 


Figure  14.  Ray  Trace  of  the  Camera  Lens  Optics. 
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Figure  16.  Actual  MTF  Measurement  for  the  Final  Camera  L«ns  Design, 


A  model  of  the  camera  lens  design  is  shown  in  Figure  17,  and  Figure  18  shows 
the  lens  system  mounted  on  the  sensor.  It  can  be  seen  that  the  cam^a,  sensor  and  focus 


servo  are  mounted  on  a  printed  circuit  board  which  houses  the  servo  controls  and  cable 
terminations.  The  model  may  be  compared  to  photographs  of  the  actual  finished  camera 
(without  housing)  in  Figure  19.  It  can  be  seen  fi*om  the  photograph  that  the  size  of  the 
system  is  dictated  by  the  package  that  houses  the  CMOS  sensor.  If  this  package  could  be 
reduced,  the  overall  size  of  the  camera  could  be  reduced. 

The  final  optical  system  has  been  designed  to  accommodate  camera  pixels  with  an 
8  fim  pitch.  The  field  of  view  is  20°  by  15°.  The  focal  length  of  the  lens  system  is  15.1 
mm.  A  comparison  of  the  initial  and  final  designs  is  shown  in  Fig.  20. 


Figure  17.  Lens  Barrel  Mechanical  Design. 


Figure  18.  Solid  Model  of  the  Final  Camera  Design.  Dimensions  are  in  mm. 
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Final  Camera  Design, 


4  Audio  Subsystem 


The  audio  subsystem  comprised  the  microphone  and  speaker.  A  number  of  alternatives 
were  investigated  and  will  be  reviewed  in  the  sections  to  follow. 

4.1  Microphones 

Three  types  of  microphones  were  evaluated  for  this  program.  The  first  comprised  a  bone 
conductivity  microphone  which  could  be  mounted  in  the  ear  as  part  of  an  earpiece,  or 
against  the  head.  The  second  comprised  a  noise-canceling  microphone  of  the  type 
typically  used  for  voice  recognition.  Since  the  digital  MP  system  was  to  use  voice 
recognition,  the  selection  of  the  microphone  was  of  critical  importance.  (The  use  of  a 
bone  microphone  would  only  be  acceptable  if  it  provided  good  performance  in  a  voice 
recognition  task.)  The  third  microphone  comprised  a  hearing  aid  transducer  mounted  in 
an  eyeglass  temple. 

Evaluation  of  microphones  was  carried  out  using  the  Microsoft  Windows98’™  Sound 
Recorder,  Dragon  Systems  Naturally  Speaking™  and  Conversa  Web™.  Tests  were 
made  in  a  quiet  lab  and  with  and  without  a  calibrated  noise  source.  A  microphone  test 
protocol  was  created  which  comprised  dictation  of  a  fixed  set  of  words  and  phrases,  and  a 
score  was  generated  corresponding  to  the  number  of  correct  times  the  system  recognized 
the  spoken  words.  The  tests  protocol  utilized  86  html  link  words  and  an  acoustic 
environment  consisting  of  speech  shaped  background  noise  at  several  calibrated  levels, 
which  is  a  standard  methodology  for  testing  hearing  aids  and  similar  audio  devices. 

The  devices  tested  comprised  (1)  an  MMIM  directional  microphone  made  by 
MicroOptical  and  integrated  in  an  eyeglass  temple,  (2)  a  noise-canceling  microphone  on 
boom  (the  VXI  Parrot),  a  bone  conduction  microphone  (the  Temco  Voiceducer). 
Microphone  arrays  which  would  seem  to  be  promising  were  not  tested  owing  to  a 
requirement  for  a  large  amount  of  digital  signal  processing. 

The  data  fi-om  background  noise  tests  are  shown  in  Fig.  21.  It  can  be  seen  that  the 
MMIM  and  VXI  microphones  have  the  best  recognition  accuracy.  However,  as  the  noise 
level  is  raised,  the  noise-canceling  boom  microphone  yields  the  best  performance.  Bone 
conductivity  microphones  show  immunity  to  noise,  but  poor  recognition  accuracy  owing 
to  the  absence  of  adequate  fi-equency  response  above  1 100  Hz  (see  Fig.  22).  While  they 
are  promising  for  future  development,  their  use  would  require  the  creation  of  specialized 
speech  recognition  acoustic  models,  which  is  beyond  the  scope  of  this  project.  For  this 
reason,  work  with  bone  conductivity  microphones  was  discontinued. 

After  we  determined  that  the  noise-canceling  boom  microphone  was  the  best  choice,  a 
number  of  noise  canceling  boom  microphones  were  evaluated.  Best  results  were 
obtained  with  a  Gentex  microphone,  which  was  used  in  the  final  MMIM  system.  Fig.  23 
shows  a  photograph  of  the  microphone. 
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Figure  21.  Bacl^ouud  Noise  Tests. 


Figure  22.  Comparison  of  Voiceducer  and  MMIM  Microphone  Frequency  Response. 
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Figure  23.  Centex  Noise  Canceling  Microphone  Used  in  the  Final  SystenL 


4.2  Ear  Phones 

A  number  of  ear  phone  speaker  devices  were  considered.  The  most  promising  comprised 
devices  comprised  (1)  a  transducer  and  pipe  that  can  be  placed  near  the  ear,  so  that 
hearing  is  unobstructed,  or  (2)  a  transducer  placed  in  an  ear  plug  so  that  very  high  noise 
levels  are  attenuated.  Either  approach  was  found  to  be  satisfactory  and  eaisy  to 
implement.  Sound  levels  from  available  transducers  were  high  enou^  to  operate  in  80 
dB  noise  environments.  Fig.  24  shows  a  photograph  of  the  transducers  used  in  final 
deliveries.  The  speaker  volume  required  for  audible  signals  above  that  noise  level  would 
pose  a  risk  to  the  user’s  hearing,  and  we  therefore  recommend  hearing  protectors  around 
the  speaker  for  environments  with  ambient  noise  levels  of  85dB  and  above^. 


Figure  24.  Packaged  Transducers  Used  in  Final  Deliveries. 


4.3  Bluetooth  Audio 


Bluetooth  audio  headsets  are  generally  avail^le  for  use  with  cellular  phones  and  for 
other  communication  applications.  At  least  one  manufacturer  (Belkin)  of  Bluetooth 
adapters  for  laptops  also  provides  software  drivers  that  enable  the  headset  to  be  used  for 
voice  command  and  control  as  well  as  dictations.  While  this  combination  is  not 
mcommended  by  any  of  the  ^eech  recognition  software  manufacturer,  in  our  tests  the 
combination  of  a  Plantronics  headset  and  the  Belkin  Bluetooth  ad^ter  gave  recognition 
rates  compamble  to  the  Gentex  microphone. 
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5  Industrial  Design 

The  industrial  effort  was  undertaken  in  close  collaboration  with  the  mechanical  and 
optical  design  process.  First,  functional  prototypes  were  fabricated  to  test  the  concept  and 
determine  the  necessary  size  and  geometry.  These  were  mounted  on  a  sturdy  adjustable 
eyeglass  frame  developed  by  Randolph  Engineering  Corporation  to  allow  the  evaluation 
of  the  concept  by  users  with  different  interpupillary  distances.  The  functional  prototype  is 
shown  in  Figure  25. 


Figure  26.  Headset  Concepts  Based  on  Safety  Glasses. 

The  next  step  in  the  design  considered  a  more  integrated  approach,  in  which  the 
components  are  fully  enclosed  by  the  frame,  as  shown  in  Figure  27.  Note  that  the  glasses 


Figure  25.  Working  Prototypes  of  Camera  and  Displays  on  an  Adjustable  IPD  Frame. 


We  believe  the  user  acceptance  is  strongly  dependent  on  both  the  comfort  and  appearance 
of  the  device.  We  envision  deployment  primarily  in  military  or  industrial  settings  that 
require  rugged  design  and  may  also  necessitate  eye  protection,  so  the  industrial  design 
approach  started  with  an  exploration  of  safety  glasses.  These  designs  provide  a  rugged, 
sturdy  platform  for  mounting  the  optical  components,  and  are  designed  to  be  comfortably 
worn  for  extended  periods  of  time.  The  look  of  the  eyewear  is  also  appropriate  to  the 
settings,  as  shown  in  Figure  26. 


retain  the  padded  nosepad  and  wide  frame  of  the  safety  glasses.  This  is  aimed  at 
maximizing  user  comfort  and  field  of  view.  This  design  approach  would  furthermore 
allow  the  user  to  wear  corrective  glasses  under  the  MMIM  headset. 


Figure  27.  Integrated  Glasses  Concept  Based  on  Safety  Glasses. 


An  alternative  approach  to  integration  is  to  fabricate  a  smaller,  more  compact  design. 
This  offers  a  more  discrete  and  pleasing  appearance  for  non-industrial  settings.  In  this 
approach,  any  prescription  correction  would  be  incorporated  into  the  electronic  headset. 
Prototypes  of  the  glasses  were  fabricated  by  Stratos  and  tested  by  Stratos  and 
MicroOptical  employees  to  evaluate  fit  and  functionality,  as  shown  in  Figure  28. 


Figure  28.  Compact  Integrated  Glasses  Concept. 


A  more  detailed  view  of  two  clip-on  concepts  integrated  into  safety  glasses  is  shown  in 
Figures  29  and  30.  For  optical  systems  integrated  into  the  eyeglass  lens,  a  more  compact 
format  close  to  the  head  was  preferred.  The  design  was  refined  several  times  as  shown  in 
Figure  31.  Finally,  more  futuristic  designs  were  considered  and  modeled,  as  shown  in 
Figure  32.  These  are  not  necessarily  practical  with  existing  technology,  but  demonstrate 
the  fixture  potential  of  the  integrated  eyeglass  approach. 
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Figure  29.  Clip-on  Integrated  Inside  Safety  Glasses. 


Figure  30.  Clip-on  Inside  Safety  Glasses.  Hand-written  CaU-Outs  Read 
(Clockwise  from  Upper  Left)  ‘Internal  Wiring’,  ‘Display  Rotates’,  and 
‘Removable  Lenses  -Can  be  Shades  or  Clear  -Can  be  Replaced  with 
Changing  Styles’. 
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Figure  31.  Evolution  of  Compact  Integrated  Eyeglass  Approach. 


Figure  32.  Futuristic  Concept  Models  for  Integrated  Headsets  Based  on  Safety  Glasses. 
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6  FIELD  TESTING 


Through  a  large  amount  of  testing  by  the  Military  Police,  MicroOptical  accumulated  a 
large  body  of  information  on  user  preferences  and  ergonomics  that  allowed  the 
development  of  a  system  that  is  easy  to  use,  is  compatible  with  legacy  systems  (software 
and  hardware),  and  that  is  comfortable  to  wear.  This  section  will  review  some  of  the 
operational  tests  of  the  system,  evaluation  trials  and  user  testing  of  the  system  at  Ft.  Polk, 
lA  fi*om  30  October  to  3  November  2000.  The  objective  of  the  trials  at  Ft.  Polk  was  to 
identify  critical  features  of  the  system  that  require  improvements  for  application  to  MP 
missions  and  tasks.  These  trials  allowed  MicroOptical  to  evaluate  the  results  of  the  design 
efforts. 


The  system  evaluation  was  focused  on  all  hardware  components  as  well  as  the  computer 
and  the  software  that  supported  its  operation.  Each  portion  of  the  system  was  evaluated 
for  ease  of  operation,  speed  and  reliability.  Fig.  33  shows  a  photograph  of  the  MPs  using 
the  Digital  MP  system.  Evaluation  and  analysis  of  the  user  tests  was  conducted  by 
James  Samson,  Senior  Human  Factors  Specialist  at  the  Natick  Soldier  Center,  in  Natick 
MA.^ 


Figure  33.  MPs  at  Ft.  Polk  Using  the  Digital  MP  System* 
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6.1  VISUAL  DISPLAY  &  EYEWEAR 


The  MMIM  system  was  evaluated  for  feel  and  comfort,  appearance/obstruction  and 
clarity/resolution  of  the  visual  display.  The  SVGA  display  previous  described  and 
shown  in  Fig,  1  was  used  in  the  evaluations.  The  clip-on  SVGA  monitor  has  see-around 
display  optics,  articulating  mounting  arm,  piano  (non  prescription)  eyeglasses,  and  is 
attached  electrically  to  the  computer  mounted  in  the  user's  load  bearing  vest  using  a  light 
weight  cable  of  approximately  125  cm  in  length.  A  clip-on  display  was  selected  to 
enable  the  users  to  make  adjustments  to  the  display  position  (Fig.  34).  There  was  also  no 
need  for  prescription  lenses  because  the  evaluators  could  mount  the  display  on  their  own 
glasses,  if  required.  None  of  the  evaluators  had  worked  with  a  head-mounted  interface 
prior  to  this  experiment. 


Figure  34.  MMBVI  SVGA  Eyewear  in  Use  by  MPs. 


Based  on  the  feedback  of  the  users,  the  visual  display  and  eyewear  were  found  to  be  very 
useful  and  easy  to  use.  Clarity  and  resolution  were  acceptable,  especially  indoors. 
Outdoors,  some  binocular  rivalry  was  observed  in  bright  light,  and  users  found  it 
necessary  to  block  light  to  their  other  eye.  This  problem  can  be  solved  by  using 
sunglasses  or  by  modifications  to  the  eyewear  itself 

Regarding  comfort,  one  of  the  principal  issues  involved  the  stabilization  and  weight  of 
the  display.  Stabilization  was  obtained  by  using  a  fabric-based  elastic  retaining  strap 
behind  the  head.  This  strap  was  effective  in  holding  the  eyewear  system  in  place,  but  it 
caused  the  nose  pieces  to  press  against  the  nose  and  resulted  in  discomfort.  It  is  believed 
that  an  integrated  glasses-based  system^  would  eliminate  this  discomfort  because  the 
weight  of  the  integrated  system  has  the  potential  to  be  lower  and  more  balanced. 

The  appearance/obstruction  of  the  display  and  eyewear  was  such  that  there  was  some 
obstruction  to  the  user’s  vision.  This  obstruction  resulted  firom  the  optical  housing  for  the 
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SVGA  display.  Had  another  display  been  used,  such  as  the  VGA  clip-on  display  (640x 
480),  this  would  not  have  been  an  issue.  Also,  the  use  of  the  integrated  eyeglass  display 
rather  that  the  hi^  resolution  clip-on  would  have  eliminated  the  occlusion  of  the  user’s 
vision.  Some  users  suggested  that  the  clip-on  display  be  mounted  on  a  helmet. 
According  to  Mr.  Sampson,  “What  complicates  the  evaluation  of  a  head-mounted  display 
for  a  user  is  the  type  of  mounting.  It  is  hard  to  separate  the  likes  and  dislikes  of  the 
display  itself  from  the  mounting  of  the  display.”  An  experiment  using  the  embedded 
eyeglass  display  would  be  of  benefit. 

6.2  HEAD-MOUNTED  CAMERA 

The  head-mounted  camera  was  found  to  be  very  useful.  It  was  found  to  be  easy  to  align 
to  the  user's  view.  The  head-mounted  camera  hardware  was  also  found  to  be  reliable. 
Regarding  function  and  utility  the  users  all  agreed  that  a  camera  is  very  useful  for  face 
recognition  and  for  capturing,  storing,  and  sharing  still  and  video  images.  Other 
comments  pertaining  to  the  camera  were  that  the  users  anticipate  a  need  for  autofocus  and 
zoom  capability,  as  well  as  ni^t  vision  capability. 

63  AUDIO  SYSTEMS 

The  audio  systems  performed  very  well.  In  order  to  reduce  ambient  noise,  the  speaker 
was  placed  near  or  in  the  ear  relatively  easily.  Some  training  and  experience  were 
required  to  properly  place  the  noise-canceling  microphone. 

The  independent  voice  control,  the  communications  (radio)  system  and  the  speakers 
worked  reliably.  The  function  and  utility  of  the  speaker  and  microphone  will  need  to  be 
tested  more  extensively  in  a  noisy  environment.  According  to  Mr.  Sampson,  “Headsets 
that  allow  hearing  of  the  enviroiunent,  as  well  as  messages  and  sounds  through  software 
and  radios  and  provide  some  hearing  protection  would  be  recommended.” 

6.4  SUMMARY  OF  FT.  POLK  USER  EVALUATION 

The  system  performed  very  well  but  there  are  a  few  design  modifications  essential  to 
bring  the  system  to  an  optimized  level  of  performance  for  hardened  military  use.  Final 
decisions  about  which  changes  will  be  made  may  require  selected  performance  tests  and 
experiments  based  on  real-world  scenarios.  Final  optimization  will  require  that  the  user, 
design  engineer  and  the  human  factors  expert  collectively  determine  the  changes,  based 
on  operational  test  and  evaluation  of  the  system,  the  software  and  the  subcomponents. 
Suggestions  and  re(X)mmendations  for  design  changes  may  then  be  reviewed  objectively 
for  utility  in  the  field. 
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7  SYSTEM  REVISION  AND  FINAL  DELIVERABLES 


The  preliminary  user  trials  at  Ft.  Polk  in  October-November  of  2000  identified  several 
areas  for  improvement.  The  users  were  pleased  with  the  potential  capabilities  of  the 
system.  However,  the  weight  of  the  system  on  the  head  and  awkwardness  in  opemting  the 
camera  were  mentioned  several  times.  In  particular,  users  foimd  it  difficult  to  aim  the 
camera  and  manually  capture  an  image  and  initiate  a  search.  The  difficulty  was  further 
exacerbated  by  the  camera  fi-ame  rate,  which  was  limited  by  the  available  bandwidth  and 
the  ability  for  image  compression/d^ompression.  The  system  revision  fiius  targeted  these 
specific  ar^s: 


•  Camera  redesign  to  incorporate  an  Omnivision  sensor  and  control  chip  set  with 
built-in  compression  for  higher  fi^e  rates. 

•  Automatic  face  recognition  capabilities  in  the  software. 

•  VGA  display  for  reduced  size,  weight  and  power  consumption. 

7.1  CAMERA 

A  primary  goal  of  the  system  revision  was  to  improve  the  camera  performance.  The  first 
issue  with  the  camera  was  a  barely  adequate  firame  rate,  which  made  it  difficult  to  aim  the 
camera  and  capture  an  appropriately  firamed  image  of  the  subject’s  fece.  The  camera 
re^onse  to  rapid  changes  in  illumination  conditions  and  backlit  scenes  was  also  targeted 
for  improvement.  At  the  start  of  this  development  program  the  selection  of  miniature 
CMOS  sensors  was  limited,  and  the  Hyundai  chip  was  the  most  appropriate  choice.  At 
the  time  the  system  was  revised,  we  again  investigated  the  available  chips  on  the  market, 
and  found  that  the  Onmivision  OV7640  sensor  and  511+  USB  controller  would  offer 
superior  performance.  This  sensor  and  controller  offer  firmware  compression  which 
would  significantly  increase  the  fiame  rate  of  the  system.  Furthermore,  readily  available 
driver  software  could  be  used  to  better  control  the  camera  performance,  without  resorting 
to  custom  software.  The  use  of  off-the-shelf  software  also  increases  the  reliability  of  the 
software  and  makes  compatibility  with  future  operating  system  and  application  software 
combinations  more  likely,  thus  enhancing  the  modularity  of  the  system.  Finally,  we 
found  that  the  auto-exposure  and  backlight  compensation  functions  of  thk  sensor  was 
more  reticle  than  that  of  the  Hyundai  chip.  The  Omnivision  is  available  in  flie  same 
package  as  the  Hyundai  chip,  so  the  mechanical  design  of  the  camera  required  only  minor 
modifications.  Since  the  sensor  is  available  in  either  full  color  or  monochrome  formats, 
several  of  the  cameras  were  fabricated  with  monochrome  sensors.  The  monochrome 
cameras  should  offer  better  light  sensitivity  and  resolution,  since  the  color  cameras  use  a 
Bayer  pattern  pixel  arrangement. 

At  this  point,  it  was  decided  to  retain  the  USB  1.1  format  of  the  camera.  While  ite 
bandwidth  is  limited  to  12Mbps,  it  is  a  commonly  available,  mature  interface  which 
maximizes  the  flexibility  of  the  system  and  permits  us  to  leverage  existing  software  and 
hardware.  With  compression,  the  hardware  offera  fimne  rates  of  up  to  15  §)s  for  full 
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VGA,  30  for  QVGA  and  GIF  format.  The  CMOS  chip  and  drivers  permit  reduced 
resolution  to  be  obtained  using  either  windowing  or  decimation,  depending  on  the  needs 
of  the  application  (field  of  view  vs.  resolution).  In  the  fixture  USB  2.0  or  Firewire  (IEEE 
1394)  will  remove  the  bandwidth  limitation  and  allow  for  higher  fi*ame  rates  or  reduced 
image  compression.  However,  neither  Firewire  nor  USB  2.0  are  available  yet  on  a 
suitable  wearable  computer. 

7.2  AUTOMATIC  FACE  RECOGNITION:  FACEIT  SURVEILLANCEtm 

Under  a  subcontract  to  MicroOptical,  Visionics  Inc.,  (now  Identix  Inc.)  was  tasked  with 
improving  the  performance  and  usability  of  the  face  recognition  fimction  of  the  software. 
This  effort  focused  on  incoiporating  automatic  face  recognition  mode  This  fimction 
relieves  the  user  fi*om  the  need  to  manually  aim  the  camera,  capture  the  image,  and 
initiate  the  database  search.  The  manual  image  capture  and  recognition  function  was  also 
retained  and  upgraded  to  the  latest  face  recognition  algorithms,  to  allow  better  certainty 
in  marginal  cases  and  for  optimal  image  capture  for  enrollment  in  the  database. 

A  screen  shot  of  the  face  recognition 
application  is  shown  in  Figure  35.  Two 
different  face  recognition  algorithms  were 
implemented  to  improve  the  system 
performance.  Using  the  Facelt  Surveillance’’'^ 
software  developers  kit  (SDK)  fi*om  Visionics, 
the  application  automatically  recognizes  feces 
in  the  video  stream  for  greater  user 
convenience.  The  FacelT  Identification 
SDK™  utilizes  manual  camera  pointing  and 
image  capture  for  higher  accuracy  of  the  face 
matching.  In  the  envisioned  use,  the  system 
would  normally  operate  in  automatic  mode, 
constantly  monitoring  the  video  input  stream 
and  attempting  to  match  feces  against  a 
database  of  known  individuals.  When  a  face  is  csptmed,  the  system  automatically 
displays  the  three  best  matches  in  the  windows  at  the  bottom  of  the  screen.  The  user  then 
has  the  option  of  dealing  with  the  individual  based  on  the  comparisons  provided.  The 
user  can  either  share  the  matches  with  his  or  her  supervisor  over  the  network,  or  switch 
to  manual  mode  for  finer  control  of  the  system.  In  manual  mode,  the  user  can  recapture 
the  image  for  better  image  quality  or  enroll  the  subject  into  the  surveillance  database. 
The  number  of  images  presented  to  the  user  was  intentionally  limited  to  three,  as  a 
wearable  environment  is  not  conducive  to  handling  complicated  cognitive  tasks. 


Figure  35.  Screen  Shot  of  Face  Recognition 
Interface  for  Digital  MP  Software. 
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7.3  VGA  CLIP-ON  DISPLAY 


As  part  of  the  software  revision,  the  graphical 
user  interface  was  optimized  to  streamline  the 
information  presented  to  the  user.  The 
interface  was  simplified  so  the  MP  can  keep 
his  attention  on  the  subject,  who,  after  all,  may 
pose  an  immediate  physical  threat.  This  also 
allows  us,  in  most  situations,  to  replace  the 
20°  horizontal  field  of  view  SVGA  display 
with  a  16°  field  of  view  VGA  display. 
Proportionally,  the  VGA  occludes  even  less  of 
the  field  of  view  of  the  user  because  it  utilizes 
a  transmissive  AMLCD  (fi’om  Kopin  Corp.) 

Figure  36.  Headset  Incorporating  VGA  uses  a  miniature  backlight  rather  than  a 

Display  with  Camera  and  Audio.  beamsplitter  for  illumination.  The  use  of  a 

smaller  display  and  optimized  software  thus 
improve  the  user’s  situational  awareness  in  two  ways:  first  by  streamlining  the 
information  for  more  efficient  reference  and  second  by  allowing  a  freer  view  of  the 
surrounding  scene.  Using  the  VGA  display  significantly  lightens  the  load  on  the  wearer’s 
face,  at  35g  compared  with  67g  for  the  SVGA  display.  A  photograph  of  the  new  headset 
is  shown  in  figure  36. 


MicroOptical’s  first  VGA  display,  the  SV-9  (originally  known  as  the  CO-3)  was 
conceived  as  an  interim  head-mounted  display  device  for  the  MMIM  program.  The  SV-9 
was  designed  to  accept  a  VGA  video  signal  from  a  wearable  PC  and  display  that  signal 
on  the  field  sequential  color  LCD  display  used  in  the  MicroOptical  HMD.  The  core 
design  was  derived  from  a  MicroOptical  VGA  demonstration  board  that  was  redesigned 
for  reduced  power  consumption  and  physical  size.  The  redesign  incorporated  state  of  the 
art  CPLDs  (Complex  Programmable  Logic  Devices),  low  power  memories  and  a  video 
input  processor.  A  board  outline  of  65  x  85  mm  provided  a  wearable  package  when 
coupled  with  a  standard  lithium  ion  rechargeable  battery.  The  complete  system  consisted 
of  a  three-board  package  to  accommodate  switches  and  connectors.  The  boxed  unit 
measured  115  x  45  x  75  mm  without  the  camcorder  battery.  The  front  panel  provided  a 
standard  female  15  pin  high  density  D  connector  for  the  VGA  source,  a  power  switch, 
and  a  switch  to  rotate  the  image  for  left  or  right  eye  operation.  The  back  panel  consisted 
of  a  machined  battery  holder,  a  power  jack  for  an  AC  adapter  and  a  switch  to  enable  a 
high  brightness  monochrome  mode.  The  circuit  provided  a  24-bit  data  path  (8  bits  per 
color  or  16,777,216  colors)  and  consumed  4  W  in  normal  brightness  mode  and  4.5  W  in 
high  brightness  mode. 


The  SV-6  was  conceived  as  a  replacement  for  the  SV-9  that  would  incorporate  more 
features  in  a  smaller,  lower  power  package.  The  size  reduction  was  primarily  made 
possible  by  advances  in  circuit  integration  and  packaging.  New  low  power,  high-density 
memories  and  high  efficiency  power  supplies  were  employed  to  reduce  overall  power 
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consumption.  A  board  outline  of  70  x  38  mm  was  chosen  to  match  the  footprint  of  the 
lithium  ion  rechargeable  battery  used  to  power  the  system.  Switches,  battery  terminals, 
LED  indicators  and  the  head  unit  connector  were  all  positioned  on  this  single  board.  A 
cable  with  a  15-pin  high  density  D  connector  on  one  end  terminated  to  wirelaps  on  the 
system  board.  The  complete  package  excluding  cables,  battery  and  strain  reliefs 
measured  78  x  42  x  17  mm.  The  SV-6  incorporated  a  3-button,  6  LED  user  interface  to 
provide  control  of  horizontal  size,  brightness,  horizontal  position,  vertical  position  and 
left  or  right  eye  flip.  The  circuit  provided  an  18-bit  data  path  (6  bits  per  color  or  262,144 
colors)  and  consumed  only  1.8  W  in  normal  brightness  mode  and  2.7  W  in  high 
brightness  mode. 

The  SV-6  added  features  while  reducing  size  and  power  consumption.  The  only 
compromise  in  performance  in  relation  to  the  SV-9  is  in  bit  depth.  Most  computer 
applications  do  not  require  24-bit  color.  Power  was  reduced  by  40%  to  55%  depending 
on  operating  mode.  Physical  size  was  reduced  by  86%.  Usability  and  interoperability 
were  dramatically  improved  by  the  SV-6. 

7.4  SYSTEM  INTEGRATION  AND  FINAL  DELIVERABLES 

The  complete  MMIM  system  consists  of  VGA  display,  USB  camera,  speaker,  boom 
microphone,  and  mouse.  A  photograph  of  an  MMIM  unit  is  shown  in  Figure  37.  Most  of 
the  systems  are  configured  with  a  USB  audio  adapter  for  maximum  reliability;  one 
system  is  configured  with  analog  audio  for  computers  without  an  available  USB  port,  and 
one  system  is  configured  with  an  off-the-shelf  Bluetooth  wireless  audio  interface  to 
demonstrate  wireless  connectivity.  Similarly,  the  mouse  may  utilize  either  a  USB  or  a 
PS/2  port  depending  on  the  computer  (adapters  are  commercially  available).  Three  of  the 
MMIM  headsets  were  configured  for  niggedized  Exponent  wearable  computers  supplied 
by  the  Natick  Soldier  Center,  by  replacing  the  standard  connectors  with  sturdier  Lemo 
connectors  and  configuring  the  VGA  display  to  receive  power  fi-om  the  wearable 
computer  battery.  These  systems  were  incorporated  into  a  military  vest  compatible  with 
the  body  armor  typically  worn  by  MP’s.  The  result  is  a  wearable  system  prototype 
suitable  for  demonstration  and  pilot  trials  in  military  settings.  A  photograph  of  the  system 


Figure  37,  MMIM  Headunit 
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is  shown  in  Figure  38.  The  rest  of  the  systems  are  configured  for  use  with  standard  laptop 
computers  or  commercially  available  wearable  computers  such  as  the  Via  or 
Xybemauf^  computers. 


Figure  38.  DMP  Vest  with  Wearable  Computer  and  MMIM  Headset. 


7.5  DEMONSTRATION  TO  THE  U.S.  NAVY  6™  FLEET 

Navy6’'’Fleet,Gaeta,IT 
July  14-18,  2003 
7.5.1  Attendees: 


Table  5:  Navy  Demonstration  Meeting  Attendees 


Office  Of  Naval  Research:  http://www.Qnr.navv.mi] 


Role  Name 


NRONRS4&T  201 

Project  Lead 

LCDRHoa  Ho,  USNR 

hoa.thi.ho@inteLcom 

NRONRS4&T  HQ  106 

C6F  NRSA  Liaison  Naval 
Officer 

CAPT(s)  Todd  Morgan,  USNR 

todd.morgan@wpc.af.niil 

C6F  Naval  Research  Science 
Advisor  (NRSA) 

Mike  Halloran 

c6f020b@c6f.navy.mil 

Role  Name 


Naval  Fleet/Force 

Technology  Innovation 

Office 

Dr.  John  Mittleman 

Naval  Fleet/Force 

Technology  Innovation 

Office,  Assistant 

Eric  Evans 

Eric_Evans@onr.navy.mil 

U*S.  Army  Soldier  System  Command:  httT3 :/7ww^Miatick .arm v.mil 


Role  Name 


Facial  Recognition  Program 

Mark  Chandler 

Manager,  Army  Technical 
Lead 

Mark.Chandlerl@us.army.mil 

MicroOptical  Engineering  Corporation:  http://vv^wv^^microoDticaLnet 


Role _  Name 


Hardware  OEM  Contractor 

Evan  Weststrate 

HMD  Manufacturer 

Electrical  Engineer 

eweststrate@microoptical.net 

7,5.2  Purpose: 


The  Navy’s  6*  fleet  is  interested  in  fulfilling  a  need  to  supplement  their  security  force 
with  a  facial  recognition  system.  Mike  Holloran,  technology  officer  with  the  6“  Fleet, 
directed  LCDR  Hoa  Ho  and  CAPT(s)  Todd  Morgan  to  investigate  the  hardware  and 
software  technologies  currently  available  for  possible  use  on  a  6*  Fleet  security  team. 
They  arranged  for  the  DigitalMP  system  demo  to  the  C6F  officers  to  evaluate  the  system. 

7.5.3  Activities: 

1 .  Initial  meetings  and  general  discussion 

2.  Demonstration  and  detail  evaluation  of  the  DigitalMP  system  to  ONR  personnel 
in  an  informal,  hands-on  setting 

3.  Demonstration  to  two  visiting  US  Marine  personnel 

4.  Preparation  of  the  C6F  staff  briefing 

5.  Active  capture  and  build  of  attendee  face  database  with  the  DigitalMP  system  in 
and  around  the  base  offices 

6.  Database  management  and  copy  to  all  wearable  machines 

7.  Staff  briefing,  five  demo,  and  Q&A 

8.  C6F  Science  Advisor  receives  and  evaluates  feedback 

9.  Preparation  of  the  C6F  Admiral’s  briefing 
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10.  Briefing  and  request  for  support  -  Vice  Admiral  Scott  Fry 

1 1 .  Closing  meetings  and  feedback  discussion 

7,5.4  Meeting  Notes: 

The  technical  demos  to  the  C6F  staff  include  a  live  face  capture/enroll  of  certain  officeis 
and  staff  memb^  and  a  mock  interrogation  and  r^ognition.  Also,  a  face  capture  and 
mcognition  with  a  known  tOTorist  database  was  demonstrated.  Many  officers,  including 
the  admiral,  were  ^le  to  don  the  vest  and  try  out  the  system  themselves. 

Several  of  the  ONR  staffers  mnained  skq>tical  of  the  Identix  software’s  robustness  with 
various  quality  and  size  databases.  Most  of  the  Q&A  fiom  the  C6F  staff  and  the  ONR 
staff  focus^  on  the  Identix  software. .  Mark  Chandler  would  arrange  a  conference  call 
with  Hentix  and  the  ONR  to  hopefully  address  ^me  of  the  unanswered  issues. 

The  primary  goal  of  the  meeting  with  the  USN  6**^  Fleet  was  accomplished  with  the 
admiral  expressing  his  support  for  confinuing  the  evaluation  of  the  a  facial  recognition 
system.  This  went  along  with  the  general  support  of  the  staff  officers,  although  the 
Navy’s  requironents  do  not  necessarily  include  a  weai^le  system.  There  is  a  need  for 
the  facial  recognition  pi^e,  but  the  final  form  is  TBD. 


34 


8  Conclusions 


The  MMIM/DMP  project  resulted  in  the  development  of  a  complete  wearable  computer 
interface,  complete  with  display,  camera  and  audio.  Two  different  displays  were 
demonstrated;  a  high  resolution,  20°  horizontal  field  of  view  SVGA  display  and  a  more 
compact  16°  VGA  display.  The  headset  also  incorporated  a  VGA  camera  based  on  a 
CMOS  chip  and  utilizing  a  standard  USB  interface,  as  well  as  high  quality  audio 
components. 


This  headset  was  incorporated  into  a  complete  wearable  system,  including  wearable 
computer,  mouse,  and  wireless  communications  capability  and  demonstrated  with 
software  designed  to  aid  the  MP  in  his  or  her  duties.  Specifically,  this  project 
demonstrated  that  in  order  to  optimize  the  user  experience,  both  the  hardware  and  the 
software  must  be  designed  with  the  end-user  and  environment  in  mind.  Situational 
awareness  is  achieved  both  by  minimizing  the  weight  and  size  of  the  equipment  and  by 
automating  tasks  to  reduce  the  cognitive  load  on  the  user.  These  principles  were 
showcased  in  a  wearable  system  providing  mobile  face  recognition  capability  (Fig.  39). 


Figure  39.  MMIM  Headset  and  DMP  Software. 
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Appendix:  System  Specification  as  of  May  28,  2003 


Table  Al.  MicroOptical  SV-6™  Display  Specifications 


Resolution 

640x480  pixels,  60  Hz  refresh 

Color  depth 

18  bit  color 

Cable  length 

4  ft  from  viewer  to  belt  worn  controls 

Field  of  view 

Approximately  16°  Horizontal,  19°  Diagonal 

Focus  range 

Adjustable,  2  to  15  ft 

Video  input 

Standard  VGA,  male  DB-15  connector 

Power  requirement 

1.6  W 

Battery 

7.2  V  Lilon  rechargeable,  4  hours  battery  life/2cell  battery 

Viewer  wei^t 

1.25  Oz 

Table  A2,  Camera  Specificai 

tions 

Sensor 

640x480  pixels,  1/3”  active  area  color  CMOS  sensor 

Frame  rate 

15  fps  @  640x480, 30  fps  @  320x240 

4  ft  from  eyeglasses  to  control  box 

Field  of  view 

Approximately  19°  Horizontal 

Optics 

19  mm  ft#  7  lens 

Focus 

Adjustable,  2ft  to  infinity 

Computer  interface 

USB  1.1 

Weidit  on  head 

1.25  Oz 

Table  A3,  Wearable  computer  system  specifications 


Total  system  weight  including  vest, 
computer,  and  batteries 

7  lbs 

Wei^t  of  head  mounted  components 

4  0z 

Processor 

500  MHz  Pentium  m™  or  Crusoe™  (min) 

Memory 

Operating  system 

Microsoft  Windows  2000™ 

Runtime 

Approximately  4-6  hours  battery  life,  hot 
swappable 

Networking 

802.1  lb  wireless  LAN 

Standard  interfaces 

VGA  out,  2  USB  1.1  ports  (min).  Keyboard, 
Mouse,  analog  audio  in/out 

Table  A4.  DMP  Software  Speciiicatf ons 


Face  recognition  software 

Facelt  Identification^**  and  Facelt  MultiFace'^**  SDK’s 
by  Identix,  Inc.  SDKs  available  for  Windows  and  Unix. 
Custom  solutions  also  available. 

Images  in  database 

Unlimited  (depoiding  on  available  computer  resources) 

Search  time  to  find  fece 

50-300  mS 

Data  base  search/match 

Up  to  60  million/minute  from  memory 

Dato  base  source  images 

Any  digitized  image  including  photographs,  video,  and 
artist  renderings 

Pose 

Optimal  performance  with  frontal  images,  accurate  up  to 
35°  off-axis.  Less  accurate  with  images  between  35  and 
90°  off-axis. 

Accuracy 

Depends  on  user  settings,  database  size  and  image  quality. 
Always  displays  top  3  matches  for  final  user  verification. 
See  FRVT2002  for  independent  government  study  of 
accuracy  for  watch  list,  one-to-many  and  one-to-one 
matching. 

Speech  recognition 

Via  Voice™  SDK  by  IBM 

User  interface 

Proprietary  GUI  developed  for  DARPA  and  US  Army 
SSCOM 

Table  AS.  Contacts 


MicroOptical  Engineering 
Corporation 

Dr.  Noa  M.  R^ing,  781.326.81 1 1x105, 
rensing@microoptical.net 

Identix,  Inc 

Dr.  Kirsten  Rudolph  Nobel,  303.551.0274, 
kirsten.nobel@identix.com 

US  Army  SSCOM 

Mark  Chandler,  508.233.4882, 
mark.chandler@Natick.army.mil 
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